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ABSTRACT 



This packet contains three papers from a symposium on 



assessing the learning organization. The first paper, "Relationship between 
Learning Organization Strategies and Performance Driver Outcomes" (Elwood F. 
Holton III, Sandra M. Kaiser), reports on a study of a new learning 
organization assessment instrument that was administered to 440 employees (53 
percent response rate) at a large nuclear power plant anticipating extensive 
changes. Reliability and validity of the instrument were established and the 
study showed that the instrument predicted 70 percent of perceived 
innovation. The second paper, "An Empirical Assessment of the Relationship 
between the Learning Organization and Financial Performance" (Andrea D. 
Ellinger, Alexander E. Ellinger, Baiyin Yang, Shelly W. Howton) assessed 
managerial responses to the Watkins and Marsick DLOQ (Dimensions of the 
Learning Organization Questionnaire) instrument along with both perceptual 
and objective measures of firm financial performance. Results suggest 
positive associations between the learning organization concept and firm 
performance. The final paper, "Translation, Validation, and Adaptation of an 
Instrument to Assess Learning Activities in the Organization: The Spanish 
Version of the Modified 'Dimensions of the Learning Organization 
Questionnaire'" (Miguel Hernandez), shares methodology and reports findings 
from a translation, validation, and adaptation study of the DLOQ. Results 
indicate that the Spanish version of the DLOQ is a valid instrument to assess 
learning activities in organizations with Spanish- speaking populations. The 
papers contain reference sections. (KC) 
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Relationship Between Learning Organization Strategies and Performance Driver 
Outcomes 



Elwood F. Holton III 
Sandra M. Kaiser 
Louisiana State University 



In this study a new learning organization assessment instrument was administered to 440 
employees (53% response rate) at a large nuclear power plant anticipating extensive changes. 

Initial factor analyses indicated 22 separate scales , all with acceptable reliabilities. Regression 
analyses showed initial evidence of criterion and discriminant validity. Hierarchical regression 
showed that instrument factors predicted 70% of perceived innovation and suggested mediated 
relationships for future structural analyses. 

Keywords: Learning Organization, Assessment Instrument, Performance Outcomes 

The learning organization literature suggests that adoption of learning organization strategies should enhance 
learning at the individual, team, and organizational levels. In addition, organizational learning should be related to 
change and innovativeness (Kaiser & Holton, 1998). And, it has been suggested that the ultimate outcome of 
organizational learning is performance improvement. However, Jacobs (1995) cautioned that research is needed to 
objectively test the claims of improved organizational effectiveness brought about through implementation of 
learning organization strategies. As Jacobs (1995) discussed, the deficiencies and the challenges include: 1) the lack 
of research supporting the claim that a causal relationship exists between learning and performance, and 2) research 
is overdue which demonstrates that the adoption of learning organization strategies lead to organization performance 
improvement. This problem has only recently begun to be addressed (Ellinger, 1998; Kaiser & Holton, 1998; 
Selden, Watkins, Valentine, & Marsick, 1998; Yang, Watkins & Marsick, 1998). 

The purpose of this study was to empirically address some of these validation issues. This field study 
developed a new instrument to assess learning organization strategies; measured perceptions of learning 
organization strategies; and, provided a preliminary test of the relationships between the organizational variables and 
the outcome measures of learning, external alignment, and innovation. The complete set of research hypotheses can 
not be listed due to space limitations but the general research questions was: Would measures of learning 
organization strategies predict learning (team, experiential, and generative) and when combined with learning, 
predict innovation and external alignment? 



Theoretical framework 

Learning organization theorists have made the claim that organizational performance effectiveness should be 
improved by adopting the features described as basic components of a learning organization (Kline & Saunders, 
1993; Kuchinke, 1995; Senge, 1992; Slater & Narver, 1993). However, they have struggled with the issue of 
empirically demonstrating relationships between the many prescribed learning organization strategies and 
performance improvement outcomes. 

Holton (1999) provided a partial answer when he addressed the meaning of organizational performance. 
He drew the distinction between performance” and performance drivers” which clarified the roles of different 
strategic outcomes. Performance was defined as the tangible products or the services provided by the organization. 
Performance drivers were defined as elements of performance that build capacity to maintain or intensify the 
individual^, processes’ or system^ ability to be effective or efficient in producing outcomes. Learning and 
innovation are examples of organizational performance drivers (Kaiser & Holton, 1998). As a set, performance and 
performance drivers explain the hypothesized cause and effect relationships in an organization^ strategy (Kaplan & 
Norton, 1996). They are both important to the successful attainment of organizational goals. In this 
conceptualization, the learning organization is an organization development strategy aimed at increasing 
organizational learning, which is defined as a performance driver. 
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While it would be ideal to measure the effect of LO strategies on performance as measured by traditional 
business outcomes such as financial indices and market share, that is often difficult to accomplish on a timely basis. 
An alternative study of the relationship between learning organization strategies and performance is to examine the 
effect of learning organization strategies on organizational performance drivers. Kaiser and Holton (1998) proposed 
that if organizational learning and innovation were conceptualized as drivers of organizational performance, the 
conceptual model shown in Figure 1 might explain the role of learning organization strategies in improving 
organizational effectiveness and performance. The model hypothesizes that adoption of learning organization 
strategies should lead to organizational learning. In turn, learning will lead to innovation in organizations that value 
creativity and innovation. Learning and innovation are predicted to act as performance drivers and should affect 
organizational performance effectiveness under the right environmental conditions. 

Figure 1 - Model of learning organization as a performance improvement strategy 
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Adapted from Kaiser & Holton, 1998. 



Method 

The site for a pilot study of this model was a large nuclear power plant. The electric power industry is currently 
undergoing extensive deregulation, similar to the changes that the telephone industry faced. The expected outcome 
of deregulation is a shift from a stable regulated utility environment to one of intense competition. Power 
companies are expected to separate the generation of electric power from the distribution function so that power 
distributors will be free to purchase electric power from any source. The prospect of a new, unstable, uncertain 
environment has led power companies to explore the learning organization as a strategy to cope with this new 
environment and enhance their competitiveness. This plant contracted with a university HRD program to conduct 
an organizational assessment. The purposes of the assessment were to evaluate organizational systems in 
relationship to learning organization characteristics and determine the scope of organizational change required to 
become a learning organization. 

Instrument Development A review of the available learning organization assessment instruments at the 
time the project began suggested that none existed which completely assessed all the hypothesized influential 
variables of organizational learning. In addition, many existing instruments had not been validated with rigorous 
psychometric procedures. Accordingly, it was decided to develop a new instrument that would be a) grounded in a 
theoretical model of organizational change and effectiveness and b) be developed using sound psychometric 
procedures. The organization^ management was open to a proposal that the consulting team develop a new 
inventory and to the addition of a research option to the project. 

The development process involved two major components. The first step consisted of a series of five focus 
groups conducted with 48 selected employees. The employees represented the following job categories: classified 
craft, organizational support (clerical), first-line supervisors, professional (engineering), and middle management 
(superintendents). Each focus group lasted for approximately 90 minutes and was held in a private 
conference/training room with only three members of the research team in attendance. 
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Next, the research team 1 used the Burke-Litwin model (Burke & Litwin, 1992) of organizational 
performance and change as the theoretical framework to develop a survey instrument. Its integrated variables 
portray the dynamic relationships in an organization system. When used with a learning lens, the Burke-Litwin 
model provided a means to integrate the organizational change literature and the learning organization literature; 
explore the important variables in organizational change and development as related to organizational learning; and 
establish a foundation upon which to assess both the process variables and the outcome variables important in 
learning organizations. 

Our approach was to examine each of the models constructs through a learning organization lens. Items 
were created to assess employee perceptions of the degree to which learning organization practices were present in 
the respondent^ organization. In addition, scales were developed to assess respondents’ perceptions of key learning 
organization outcomes. It is important to note that the employee focus groups were used to provide a reality check, 
but were not the primary source of the items. The instrument was developed primarily by integrating organizational 
change and learning organization theory. 

The Kaiser-Holton model shown in Figure 1 does not attempt to describe the relationship between the 
learning organization strategies, but only the relationship of the strategies to learning and performance as an 
organizational development system. However, the Burke-Litwin model hypothesizes the relationship between the 
organizational factors that are the targets of the learning organization strategies. These two models used in concert 
were hypothesized to represent the relationship between the learning organization strategies, learning, and 
performance. 

Learning Organization Inventory The Learning Organization Inventory contained a total of 212 items 
designed to tap employees’ perceptions about various learning practices, beliefs, behaviors, and attitudes. This 
instrument has also been identified elsewhere in the literature under the name Assessing Strategic Leverage for the 
Learning Organization (ASLLO) (Gephardt, Holton, Marsick & Redding, 1997; Holton & Kaiser, 1997). The 
instrument was divided into nine construct domains: organization as a whole, work groups, business strategies and 
plans, general beliefs, senior management, managers/supervision, organizational support systems, organizational 
structure, and what is rewarded and expected. Each item used a six-point Likert rating scale (1= not true, 2= mostly 
not true, 3=somewhat not true, 4=somewhat true, 5=mostly true, 6=true). A demographic component addressed the 
respondents ’work group, job category, level of attained education, and time of service with the company. 

Sample The inventory was sent via intercompany mail to all 828 employees in the organization. A pre- 
addressed envelope was provided with instructions to use company mail to return to the surveys to the university. 
No company personnel handled the completed surveys. A total of 443 inventories were returned with 440 usable for 
a usable return rate of 53%. 

It became apparent early in the collection phase that there was a strong reluctance on the part of craft 
employees to participate in the assessment process. Suspecting that lack of time during the workday may have been 
a reason for the low participation, arrangements were made to provide employees an on-site opportunity to complete 
the inventory. This added only a few more inventories, bringing the number completed by craft employees to a total 
of 82 inventories, or 18.6% of the sample. Thus, craft employees are under-represented in the sample. A similar 
under-representation occurred with Senior Management; only seven of twelve responded. This lack of participation 
is perplexing because the learning organization initiative began with Senior Management. Professional/Technical 
employees were over-represented, with 222 respondents or 50.5% of the sample. 

Analysis Exploratory factor analysis was deemed the most appropriate approach for scale development. 
While the instrument had a reasonably strong theoretical foundation, the integration of two theoretical bases made 
confirmatory factor analysis premature. Also, the large number of inventory items compared to the number of 
subjects prevented an exploratory factor analysis of all 212 items simultaneously. The decision was made to 
examine the items for underlying factors using the nine construct domains defined on the inventory. Each construct 
domain was a separate section on the questionnaire and had separate instructions. Items in each category were 
factor analyzed by principal axis factor analysis with oblique rotation (direct oblimin). Principal axis (common 
factor) analysis and oblique rotation are generally considered more appropriate when the purpose is to identify latent 
constructs for subsequent theoretical interpretation (Hair et al., 1998). Items loading at .40 or higher were retained. 
Item-respondent ratios ranged from 1 1.0 to 48.9, all in excess of the desired 10: 1 ratio (Hair, et al., 1998) 

It is clearly possible that a more parsimonious scale structure would emerge if the factor analysis were 
conducted with a larger sample and all items entered simultaneously into the analysis. However, the instrument 
specifically identified nine separate domains for which scales should be identified. Therefore, the current analysis is 
conceptually defensible for initial analyses, though these results should be considered preliminary. 

Initial determination of the number of factors to retain was done using the eigenvalue equal to one 
criterion. However, this criteria is not always the most appropriate when the purpose of the analyses is to discover 
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latent organization entry constructs and not simply to reduce the number of variables used in subsequent analyses. 
Interpretable factors with low eigenvalues could capture important latent organizational entry constructs. Thus, the 
analysis also included examination of factor structures at least one above and below the number indicated by the 
eigenvalue equal one criteria for each set of items as well as examination of scree plots. 

Multiple regression and hierarchical multiple regression analysis (Cohen & Cohen, 1983) was used to 
evaluate to determine predictors of perceived learning and innovation outcome variables. Hierarchical regression 
analysis is particularly appropriate in early stages of theory development as it not only evaluates predictive power 
but can also suggest possible meditated relationships for future path or structural equation modeling analysis. 



Results 

Demographics The sample consisted of employees who were predominantly working in professional/technical jobs 
(50.2%); had some college (28.9%) or a bachelor's degree (25.9%); and had been employed by the company 3-10 
years (47.3%) or 11-15 years (33.9%). The inventory also had a choice of 23 work groups from which employees 
were to select the one most appropriate for them. The single most frequently selected work group was Site 
Engineering with 59 employees or 13.4% of the total respondents. 

Factor Analysis Results Factor analyses identified 22 separate scales, all with acceptable reliabilities (see 
Table 1). In 7 of the 9 domains the factors retained were those that were a) identified when the eigenvalue equal one 
criterion was used, b) had a Cronbach's alpha reliability of .70 or higher (Nunnally & Bernstein, 1994) and c) had at 
least two items. It should be noted that Gorsuch (1983) recommends not retaining any factors with less than three 
items. However, given the early development stage of this instrument, it was decided to retain one factor with two 
items that appeared theoretically sound. Two domains (manager/supervisor, rewarded and expected) did not result in 
clean interpretable structures initially. In both cases, fewer factors than indicated by the eigenvalue equal one 
criterion were deemed appropriate. This is consistent with evidence that this criteria can result in over factoring, 
while the scree test may yield fewer factors (Gorsuch, 1997). The scales that emerged from the factor analysis were 
used to interpret the organization's data and complete the objectives of the project. 
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Table 1 

Instrument factors, reliabilities, and definitions 



Organization Factors -24 items 



Innovation 


9 


.90 


ability of the organization to adopt and/or create new ideas and to implement these 
ideas in the development of new and better products, services, and work processes 


Motivation/Engagement 


5 


.79 


levels of organizational commitment and job involvement as expressed by the 
work effort and behaviors of employees 


Generative Learning 


7 


.81 


organization's ability to understand business goals and problems and make core 
changes needed to better attain stated objectives 


Learn from experience 


3 


.84 


organization's ability to learn from experiences, whether the experiences are 
considered successes of failures, and to draw on the knowledge learned to make 
better decisions or business improvements 


External Alignment 


5 


.81 


ability of the organization to understand its relationships with and the needs of its 
business environment, markets, suppliers, and customers in order to remain 
competitive and viable 


Internal Alignment 


5 


.84 


level of organization's integration of goals, functions, roles, work efforts, problem- 
solving and decision-making, in order to increase organizational effectiveness 



Work Group Factors - 9 items 


Team Learning 9 


.90 


ability of workgroups to acquire, interpret, and share knowledge in order to 
enhance the group level learning and work practices 


Business Flans and Strategies -4 5 items 


Knowledge Creation 7 

External Monitoring 2 

Systems Thinking 6 


.89 

.82 

.83 


extent to which the organization seeks to acquire, disseminate, and interpret 
information to establish an organizational knowledge-base to improve 
organizational performance 

level of organizational efforts to be aware of business and industry trends and 
forces which affect organizational effectiveness 

degree to which the organization and its members both recognize and act from an 
organizational systems perspective 



General Beliefs -14 items 



Knowledge 
Indeterminacy 
Learning Latitude 
(Risk-taking) 
Organizational Unity 


5 

4 

5 


.80 

.75 

.81 


belief that knowledge is not fixed and any individual may be a source of 
knowledge, while no one person can know all things 

extent to which organizational members perceive that they can take risks to learn 
and try new things 

belief that all organizational members are working toward recognized common 
goals for the benefit of the organization 


Senior Management Factors - 


12 items 




Leader Support for 


13 


.94 


perceived level of strong, visible commitment by senior management to the values 


Learning 






and actions necessary in a learning environment 


Manager/Supervisor Factors - 


-26 items 




Management Learning- 


13 


.94 


extent to which employees’supervisors take actions to promote and enable 


Support Practices 






learning to occur 


Management Learning- 


5 


.88 


extent to which supervisors' actions motivate employees to learn and develop both 


Motivation Practices 






as individuals and as groups 


Management 


5 


.89 


extent to which supervisors take action to enhance job performance and 


Performance- 






effectiveness 


Effectiveness 








Management Logistical 


3 


.84 


extent to which supervisors create work situations that enable effective job 


Provision/Support 






performance 


Practices 








Aspects of the Organization - 1 3 items 


Supportive 


13 


.90 


perceived ability of various organizational systems (communication system, 


Organizational Systems 






information system, human resource system) to support organizational learning 


Way the Organization is Structured -5 items 


Facilitative Structures 


5 


.77 


extent to which the organizational structures provides access to individuals and 








groups both inside and outside the organization for information sharing 


Climate - 22 items 


Generative Learning 


12 


.93 


extent to which the work climate encourages continual learning that challenges 


Climate 






organizational norms 


Promotive Interaction 


10 


.85 


degree to which individuals act to encourage and facilitate each others’ efforts to 








grow, perform, and achieve success 
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The 22 scales of this inventory assessed Burke-Litwin's transformational and transactional variables affecting 
change, learning and growth. Table 2 shows transformational and transactional variables suggested by the Burke- 
Litwin model, and what are believed to be the appropriate scales from the assessment instrument which provide a 
measure of each of these organizational variables through a learning lens. 

Table 2 

Comparison of Burke-Litwin model and instrument scales, along with pilot site results. 



Burke-Litwin Model Construct 


Assessment Instrument Scale 


Pilot Site Means 
(1-6 scale) 


Transformational Dynamics of Learning: 


External Environment 


Not assessed: influences external to the organization 




Leadership 


Leader Support for Learning 


3.74 


Mission and Strategy 


Knowledge Creation 


4.02 




Internal Alignment 


3.90 




Systems Thinking 


3.81 


Organizational Culture 


Knowledge Indeterminacy 


5.04 




Learning Latitude (Risk-taking) 


3.60 




Organizational Unity 


5.05 


Transactional Dynamics of Learning: 


Management Practices 


Management Learning Support 


4.29 




Management Learning-Motivation 


3.45 




Management-Performance-Effectiveness 


4.02 




Management-Logistical Provision/Support 


3.57 


Structure 


Facilitative Structure 


3.33 


Systems: Policies/Procedures 


Supportive Systems 


3.46 


Work Unit Climate (rewarded and 


Generative Learning Climate 


3.80 


expected) 


Promotive Interaction 


3.62 


Motivation 


Motivation/Engagement 


4.25 


Task Requirement & Individual 


Not Measured 




Skills/Abilities 






Individual Needs/Values 


Not Measured 




Outcomes 


System level Learning Outcomes 


Learn from experience 


4.24 




Team Learning 


4.25 




Generative Learning 


3.20 


Performance Drivers 


Innovation (perceived) 


4.14 




External Alignment 


4.13 



Descriptive Results Table 2 also shows the mean scale scores for the pilot site. Because this was the first 
use of this instrument, no norms were available to which these scores could be compared. Analysis of variance 
(ANOVA) was conducted for each scale within each of the four demographic classifications: work group, job 
category, education, and time with the company. Significant differences were found within the job, education, and 
service time categories. For job category, organizational support employees reported higher scale means than other 
groups on 1 1 scales. Classified craft/operators also reported some differences. A two-way ANOVA was used to 
explore the possibility that differences found among the job categories might be due to an interaction with 
educational level. No interaction effect was found. For educational level, employees with only a high school 
education reported significantly higher scale means on 17 of 22 scales. Finally, for length of service, employees 
with more than 15 years of service reported lower scale means on 6 scales. 

Analysis of Outcomes Five outcome scales were identified based on the conceptual framework underlying 
the instrument. It is important to note that these outcomes were all respondent perceptions of outcomes, not 
objective measures. As shown in column one of Table 3, they represent four system level outcomes (#1 - 4) and one 
team/group level outcomes (#5). The correlations among these outcome variables are also shown in Table 3. All of 
the correlations were statistically significant and moderately strong. 
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Table 3 



Correlations among outcome scales 



1 


2 


3 


4 


1 -Innovation 










2-Generative Learning 


.565 








3-Leaming from Experience 


.597 


.500 






4-Extemal Alignment 


.684 


.508 


.615 




5-Team Learning 


.706 


.547 


.578 


.561 



Next, measures of learning organization characteristics were regressed on each of the five dependent 
variables to gain initial insights into the predictive power of the variables as hypothesized in the instrument's 
conceptual frame. Table 4 shows the scales with significant standardized betas for each dependent variable, along 
with the R 2 for each model. The R 2 were high, ranging from .50 (experiential learning) to .71 (innovation). While 
one must interpret this finding cautiously due to the possibility of common method variance, the magnitude of the 
variance explained nonetheless provides initial evidence of criterion validity for these scales. Furthermore, the 
significant predictors varied with no predictor appearing in all equations. Knowledge creation and internal 
alignment appeared in four equations, management performance effectiveness and motivation/engagement in three 
equations, and the rest in two or fewer equations. This lends support to the discriminant validity of the scales. 

Kaiser and Holton (1998) proposed that learning organization strategies such as those measured by these 
scales improve performance by increasing learning which in turn increases innovation and external alignment, 
leading to improved performance outcomes under conditions of environmental uncertainty. Thus, a second analysis 
was conducted using hierarchical regression to determine the extent to which the predictor variables added to the 
variance explained in each dependent variable. For space reasons, only the analysis for innovation is discussed here 
(see Table 6). 

Table 4 

Regression analysis results for perceived performance driver outcome variables 



Assessment Instrument 


Experiential 


Team 


Generative 


Innovation 


Ext Align 


Scale 


Learning 


Learning 


Learning 






Leader Support for 
Learning 

Knowledge Indeterminacy 
Learning Latitude (Risk- 
taking) 

Organizational Unity 
Systems Thinking 




.15 




.14 


.12 


External Monitoring 








.23 


.17 


Knowledge Creation 


.20 


.26 


.15 


.11 




Mgt. Learning Support 
Mgt. Learning-Motivation 




.15 








Mgt. Performance- 
Effectiveness 




.18 


-.13 


-.11 




Mgt. Logistical 
Provision/Support 










-.14 


Internal Alignment 
Facilitative Structure 


.36 


.22 


.25 




.18 


Supportive Systems 
Generative Learning 
Climate 

Promotive Interaction 


.15 


.13 


.36 






Motivation/Engagement 




.13 


.19 


.12 




Experiential Learning 








.12 


.20 


Team Learning 
Generative Learning 








.16 




R 2 


.50 


.69 


.59 


.71 


.58 


(Adjusted R 2 ) 


.48 


.68 


.58 


.69 


.56 
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Each set of learning organization organization strategy variables was entered separately in the analysis 
(steps 1-8) with learning outcome variables entered in step 9. Key findings included: 1) leader support for learning 
was a significant predictor in all steps, though decreasing in magnitude, 2) knowledge 

Table 6 

Hierarchical Regression Results for Perceived Innovation 



Assessment Instrument Scale 


Step 1 


Step 2 


Step 3 


Step 4 


Step 5 


Step 6 


Step 7 


Step 8 


Step 9 


Leader Support for Learning 


. 68 ** 


44 ** 


. 20 ** 


. 20 ** 


.17** 


.16** 


.15** 


.13** 


.14** 


Knowledge Indeterminacy 




.15** 


. 11 ** 


. 10 ** 


. 10 ** 


. 10 ** 


. 10 ** 


.08** 


.07 


Learning Latitude (Risk-taking) 




.29** 


.01 


.01 


-.10 


-.02 


-.05 


-.07 


-.08 


Organizational Unity 




-.03 


-.02 


-.03 


-.03 


-.03 


-.03 


-.02 


-.02 


Systems Thinking 






.18** 


.16** 


. 10 ** 


.10 


.09 


.09 


.09 


External Monitoring 






.25** 


.24** 


.24** 


.23** 


.23** 


.24** 


.23** 


Knowledge Creation 






.27** 


.27** 


.23** 


.23** 


.19** 


.18** 


. 11 * 


Mgt. Learning Support 








.14* 


.12 


.12 


.09 


.08 


.05 


Mgt. Learning-Motivation 








-.04 


-.08 


-.08 


-.09 


-.06 


-.04 


Mgt. Performance- Effective ness 








-.05 


.06 


.06 


.05 


.03 


.01 


Mgt. Logistical Provision/Support 








-.13** 


-. 12 * 


-. 12 ** 


-. 12 * 


-. 11 * 


-. 11 * 


Internal Alignment 










.16** 


.15** 


.16** 


.14** 


.06 


Facilitative Structure 










.06 


.05 


-.02 


-.02 


-.02 


Supportive Systems 












.04 


-.01 


.01 


.01 


Generative Learning Climate 














.16** 


.15* 


.11 


Promotive Interaction 














-.06 


-.03 


-.05 


Motivation/Engagement 
















.14** 


. 12 ** 


Experiential Learning 


















. 12 ** 


Team Learning 


















.16** 


Generative Learning 


















-.02 


R 2 

(Adjusted R 2 ) 


.46 

(.46) 


.53 

(.53) 


.65 

(.64) 


.66 

(.65) 


.67 

(. 66 ) 


.67 

(. 66 ) 


.68 

(. 66 ) 


gj © 

OO 

'w' 


.71 

_ (-69) 


R 2 Change 


- 


.072** 


. 120 ** 


.009* 


. 012 ** 


.001 


.007* 


© 

© 

• 

• 


.017** 



* (p<.05; ** (p<01 

indeterminancy was also significant until learning outcomes were entered in the last step, 3) external monitoring and 
knowledge creation remained significant after entering the equation, 4) generative learning was not a significant 
predictor. 

Conclusion 

While these results are only an initial test of the instrument, they do offer initial evidence of construct, criterion and 
discriminant validity for the instrument. This research demonstrates that a more comprehensive learning 
organization assessment instrument could be developed with rigorous psychometric procedures. Significant portions 
of the variance in perceived learning were explained by learning organization constructs. Furthermore, significant 
portions of the variance in perceptions of innovation and external alignment, which are key performance drivers, 
were explained by learning organization constructs. While the instrument clearly needs further testing and 
refinement, the results are encouraging. In addition, the theoretical foundations of the instrument along with the 
pilot site test offer encouraging prospects for a more complete learning organization theory and empirical tests of 
that theory. 

One limitation of this study is that the organization studied was not one that was known to be a complete 
learning organization. It did have some aspects of a learning organization in place as a result of prior development 
programs such as total quality and team oriented work. One interpretation of the regression results suggesting that 
constructs suggested by learning organization theory to advance learning, but not found to be significant predictors 
in this study, is that learning organization theory was not supported. Alternatively, these results may suggest that the 
organization needs to develop these areas of the organization as they represent current weaknesses. Further research 
in different organizations is needed to be certain which is the proper interpretation. 

*Note: Special thanks to Victoria Marsick, Martha Gephart, and John Redding who assisted with the early phases of 
instrument development. 
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The concept of the learning organization has received considerable attention in the literature as 
firms are increasingly encouraged to leverage learning to gain competitive advantage. However , 
few empirical studies have examined the relationship between the learning organization concept 
and firm performance. To assess this association , managerial responses to the Watkins and 
Marsick DLOQ© instrument were obtained along with both perceptual and objective measures 
of firm financial performance. Results suggest positive associations between the learning 
organization concept and firm performance. 



Keywords: Learning Organization, Firm Performance, Financial Performance 

Scholars have suggested that learning may be the only source of sustainable competitive advantage. Considerable 
attention has been devoted to the concept of learning organizations in the past several years. Yet, there have been 
more thought papers on why learning matters than on the processes required to building learning organizations 
and their potential impact on firm performance. Despite the numerous accounts and suggestions that discuss why 
the learning organization concept presumably works, few concrete studies exist that clarify how it works to achieve 
performance improvement (Kaiser & Holton, 1998). Jacobs (1995) suggests that there are little data supporting the 
claim that performance improvement is directly related to the adoption of practices associated with the learning 
organization literature. 

Other researchers are concerned that the concept of the learning organization is being ‘bversold as a near- 
universal remedy for a wide variety or organizational problems (Kuchinke, 1995, p. 307). Because many of the 
contributions on the learning organization to date have been descriptive and prescriptive, the need for empirical 
research on this concept has been articulated by several scholars (Altman & lies, 1997; Jacobs, 1995; lies, 1994; 
Leitch, Harrison, Burgoyne, Blantem, 1996). In particular, one of the major research challenges is to establish the 
relationships between characteristics of the learning organization and organizational performance ( lies, 1994; 
Leitch, Harrison, Burgoyne, Blantem, 1996). 

Recent studies have begun to establish a research base that examines the dimensionality of the concept of 
the learning organization (Watkins, Yang & Marsick, 1997; Yang, Watkins & Marsick, 1998). However, if firms 
are to create learning organizations by focusing on the implementation of practices and processes that promote 
learning at the individual, team, and organizational levels, the linkages to improved organizational performance 
must be more firmly established. Therefore, the current research examines the relationship between the 
dimensions of the learning organization and financial performance utilizing both perceptual measures of firm 
performance and secondary financial data drawn from the COMPUSTAT and the Stern Stewart Performance 1000 
financial databases. Assessing the relationship between the learning organization concept and objective measures 
of firm financial performance represents an empirical methodology that has not been employed to date. The 
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research also attempts to further validate the Watkins and Marsick Dimensions of the Learning Organization 
Questionnaire © (DLOQ) in a business context with mid-level managers as respondents. 



Theoretical Framework 

The theoretical grounding for this research is the Watkins and Marsick conceptualization of the learning 
organization (1993, 1996a, 1996b). The foundation of the Watkins and Marsick perspective is that the design of a 
learning organization depends upon seven complementary action imperatives: (1) create continuous learning 

opportunities; (2) promote inquiry and dialogue; (3) encourage collaboration and team learning; (4) establish 
systems to capture and share learning; (5) empower people toward a collective vision; (6) connect the organization 
to its environment; and, (7) use leaders who model and support learning at the individual, team, and organizational 
levels. 



Research Questions 

The study addresses the following research questions: 

(1) . To what extent does testing the DLOQ instrument in a different context offer further validation for 

Watkins and Marsickfc instrument? 

(2) . What is the relationship between the seven dimensions of the DLOQ instrument and the perceptual 

organizational outcome variables as defined by financial and knowledge performance? [The DLOQ asks 
respondents for perceptual assessments of various measures associated with financial performance and knowledge 
performance]. 

(3) . What is the relationship between the seven dimensions of the DLOQ instrument and objective 

organizational outcome variables as defined by four secondary measures of financial performance? [return on 
equity (ROE), return on assets (ROA), Tobinfc q, and market value-added (MV A)] 



Research Design 

A mail survey methodology was employed to address the research questions. The procedures used to design the 
sampling frame correspond to those outlined by Dillman (1978). 

Sample. A random sample of 400 mid-level managers at U.S. manufacturing firms was obtained from the 
Council of Logistics Management Membership listing. The selection of logistics managers as key respondents for 
this study was based upon the increasing role of supply chain management as a key element in corporate strategies 
that focus on service for the provision of superior customer value (Christopher & Ryals, 1999; Poirer, 1999; Stank, 
Daugherty & Ellinger, 1998). The logistics function must receive, assess, and act upon so much important 
customer feedback and data that firms with effective learning processes may be better equipped to provide their 
customers with better service. In addition, logistics managers must continuously interact with other corporate 
functions. Accordingly, logistics managers’ perceptions of their firms’ learning behaviors represent a unique 
platform from which to examine the dimensions of the learning organization concept and their impact on 
performance. 

Potential respondents’ firms were screened for the availability of secondary data for their firms on the 
COMPUSTAT database. Respondent firms from the random sample for whom data was not found on the 
COMPUSTAT database were replaced. Prenotification of prospective respondents is believed to increase response 
rates (Fox, Crask, & Kim, 1988). Therefore, each of the managers in the sampling frame was contacted by 
telephone to solicit his/her participation in the study. Additionally, since type of postage, the sponsorship of a 
university, and monetary incentives are also believed to be influential factors for increasing response rate (Fox et 
al., 1988), the initial mailing included prepaid return postage, a personalized letter on university letterhead, and a 
$2 bill as an incentive to respond. Non-respondents were contacted with a follow-up letter two weeks after the 
initial mailing. A total of 262 surveys were mailed and 208 completed surveys were returned resulting in a usable 
return rate of 52%. 
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Instrumentation. The DLOQ instrument (Watkins & Marsick, 1993, 1996a, 1996b) was used for this 
study. The seven dimensions in the Watkins and Marsick instrument are measured by 43 items. Previous research 
using this instrument has been conducted by Watkins, Yang and Marsick (1997), Yang, Watkins, and Marsick 
(1998), and Yang, Watkins, and Marsick (forthcoming). Accordingly, several stages of empirical research have 
assessed the psychometric properties of the DLOQ. These analyses suggest that the seven dimensions have 
acceptable reliability estimates (coefficient Alpha ranges from .75 to .85). The seven factor structure was also 
found to fit the empirical data reasonably well (Yang, Watkins & Marsick, 1998). The previous examinations of 
the DLOQ utilized non-random samples of 1 16 and 469 respondents respectively from multiple organizations in 
instrument development and validation studies (Watkins, Yang & Marsick, 1997; Yang, Watkins & Marsick, 
1998). 

Perceptual Performance Measures. The two performance outcome measures on the DLOQ instrument, 
Financial Performance and Knowledge Performance, ask respondents to indicate their assessments of the 
organization^ current performance when compared to the previous year. The first performance variable, Financial 
Performance, is assessed in the following areas: return on investment, average productivity per employee, time to 
market for products and services, response time for customer complaints, market share, and the cost per business 
transaction. The second performance variable, Knowledge Performance, is assessed in the following areas: 
customer satisfaction, the number of suggestions implemented, the number of new products or services, the 
percentage of skilled workers compared to the total workforce, the percentage of total spending devoted to 
technology and information processing, and the number of individuals learning new skills. 

Secondary Financial Performance Data. A database consisting of secondary measures of financial 
performance for the respondent organizations in the study was created with data obtained from the 1998 
COMPUSTAT the Stern Stewart Performance 1000 financial databases. Specifically, four measures were chosen 
to obtain a comprehensive view of firm financial performance: return on equity (ROE), return on assets (ROA), 
Tobink q, and market value-added (MV A). The MVA data obtained for this study is from the Stern Stewart 
Performance 1000 and is 1998 data for 1,000 firms. MVA data is quoted in a dollar amount for each firm. Since 
the research here includes firms with varying sizes, MVA is standardized by total assets, a proxy for firm size. The 
ROA and ROE measures are from the COMPUSTAT database and are listed for each company under the data 
items of ROA and ROE. A proxy for Tobink q was calculated using a method suggested by Chung and Pruitt 
(1994). All of the data necessary to calculate the proxy were obtained from the COMPUSTAT database. 



Data Analysis 

To address the first research question, confirmatory factor analysis (CFA) was used to assess the dimensionality 
and validity of the DLOQ in a different context. The CFA was conducted with LISREL 8 (Joreskog & Sorbom, 
1989, 1993a, 1993b). CFA is a procedure that examines the construct validity of an instrument with pre-specified 
dimensions. In this study, we are particularly interested in assessing whether, in a business context, the DLOQ 
measures constitute an adequate measurement model for the learning organization concept 

Two measurement models were examined to assess the DLOQ in the new context: one for all 43 learning 
organization items, and another for a reduced set of 21 items. An earlier study conducted an extensive series of 
exploratory and confirmatory factor analyses where the 43-item scale representing the seven dimensions of the 
learning organization was reduced to a more parsimonious 21 -item scale. The resulting 21 -item, seven-construct 
model yielded superior fit indices than the original 43-item model (Yang, Watkins & Marsick, 1999). 

Canonical correlation was selected to assess associations between dimensions of the learning organization 
and perceptual and objective measures of firm performance (research questions 2 and 3). Canonical correlation is 
a technique for examining the association between two sets of variables (Stevens, 1996). The underlying principle 
is to develop a linear combination of each set of variables (both independent and dependent) in a manner that 
maximizes the correlation between the two sets. 

Canonical correlation was chosen over structural equation modeling (SEM) as a more appropriate 
statistical technique with which to explore an omnibus impact of the dimensions of the learning organization on a 
set of financial performance indicators. The objective was to assess overall effects of the learning organization 
concept on firm performance rather than causal relationships. 

SEM also requires more proven measures to be used in data analysis than were available for secondary 
financial performance. As no one measure is able to completely describe all aspects of a firmk condition, it is 
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important to collectively examine several different measures of performance (Brigham 1995; Peterson 1994). We 
selected a combination of traditional accounting and value-added indicators to reflect an adequate, but nevertheless 
exploratory, measure of the concept of financial performance. The canonical correlation analysis was performed by 
MANOVA procedure using the SPSS statistical package (Norusis, M. J./SPSS Inc., 1990). 

Results of the Confirmatory Factor Analysis . Table 1 offers a comparison of the fit indices for the two 
measurement models. Six criterion indices were chosen to evaluate the fit between the proposed measurement 
model and that generated from the sample. The indices selected were the chi-square (% 2 ) test, Joreskog and 
Sorbomfc (1989) goodness of fit index (GFI) and goodness of fit index adjusted for degree of freedom (AGFI), 
Bentlerfc (1990) comparative fit index (CFI), Bentler and Bonettfc (1980) non-normed fit index (NNFI), and 
Steigerfc (1990) root mean square error of approximation (RMSEA). 

The GFI and AGFI reflect the proportion of joint amount of data variance and covariance that can be 
explained by the measurement model being tested. The NNFI is a relative fit index that compares the model being 
tested to a baseline model (null model), taking into account the degrees of freedom. The CFI indicates the degree 
of fit between the hypothesized and null measurement models. 

The RMSEA represents a real advance in the evaluation of model fit from both a statistical and a 
conceptual viewpoint. Browne and Cudeck (1993) argue that because theoretical models are at best 
approximations of reality, the null hypothesis for any measurement/structural equation model (i.e., the 
conventional chi-square test that the data fits the model perfectly) will rarely be true. Rather than testing the null 
hypothesis of exact fit between the covariance matrix of sample and that of model for population, RMSEA 
establishes a hypothesis of close fit between the model and population. RMSEA values of .05 or less indicate a 
very close fit between sample and theoretical model, accounting for degrees of freedom. Values less than .08 
reflect reasonably well fitting models (Browne & Cudeck, 1993). 



Table 1. Fit Indices for Measurement Models of the DLOQ 



Fit Index 


Measurement Models 




43 Items 


21 Items 


x 2 


3886.576 


328.544 


df 


839 


157 


X 2 /df 


4.632 


2.093 


RMSEA 


.132 


.073 


RMSR 


.083 


.053 


GFI 


.616 


.870 


AGFI 


.567 


.808 


NNFI (TLI) 


581 


.914 


CFI 


.611 


.935 



The CFA results for the new sample add further credence the dimensionality proposed by the DLOQ 
instrument developers. Specifically, the 43-item model with seven underlying dimensions fits the data only 
moderately, while the reduced seven- factor 21-item model forms a reasonable measurement model (RMSEA < 
.08). The proposed seven dimensions of the learning organization account for eighty seven percent of item 
variances and covariances (GFI = .87). Both the NNFI and the CFI are above the .90 level, indicating that the 
seven-dimension structure fits the data very well in comparison to the baseline measurement model. 

Table 2 presents reliability estimates for the full and reduced set of items for the DLOQ. The reliability 
estimates do not decrease substantially when the number of items is reduced by half (with the exception of 
Continuous Learning). These findings suggest that the full and reduced set of DLOQ measures continue to 
demonstrate acceptable reliability in the new context. 
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Table 2. Reliability Estimates for the DLOQ Measures 



Subscale 


Original 43 Items 


Reduced 21 Items 


Number of 
Items 


Coefficient 

Alpha 


Number 

Items 


of Coefficient 
Alpha 


Continuous Leaning 


7 


.81 


3 


.60 


Dialogue & Inquiry 


6 


.86 


3 


.78 


Team Learning 


6 


.85 


3 


.77 


Embedded System 


6 


.85 


3 


.75 


System Connection 


6 


.87 


3 


.80 


Empowerment 


6 


.84 


3 


.72 


Provide Leadership 


6 


.89 


3 


.87 


Financial Performance 


6 


.75 


3 


.68 


Knowledge Performance 


6 


.80 


3 


.71 



Results of the Canonical Correlation Analysis. Table 3 shows the results of the canonical correlation 
analyses between dimensions of learning organization and the perceptual and objective financial outcome 
variables. Because our primary purpose was to examine the associated variability between the two sets of variables, 
rather than the structure of the variables, our discussion focuses on the overall effects of the canonical correlation 
analyses. 

The multivariate tests suggest a statistically significant relationship between the seven dimensions of the 
learning organization and the two perceptual outcome variables: Financial Performance and Knowledge 

Performance (p < .001). Effect sizes of the canonical correlation range from .246 to .312, indicating that more 
than a quarter of the variability in the respondents’perceptions of organizational performance can be accounted for 
by the seven dimensions of the learning organization. 

The canonical correlation between the seven dimensions of the learning organization and the four 
secondary measures of financial performance (ROE, ROA, Tobinfc q, and MV A) is also statistically significant (p 
< .05). Moreover, different multivariate statistics reveal consistent effect size, ranging from .104 to .108. Thus, 
more than ten percent of the variance in the four financial indicators can be explained by the dimensions of the 
learning organization measured on the DLOQ. 

In summary, our analyses offer support for the validity of the DLOQ instrument using a random sample of 
respondents in a business context. In addition, the results suggest a positive association between the learning 
organization concept and firm performance. 



Table 3. Multivariate Tests of Significance for Canonical Correlation 



Test Name 


Value 


Approx. F 


Hypoth. df 


Error df 


Sig. of F 


Effect Size 


Test for Two Perceptual Outcome Variables 










Pillais 


.492 


6.611 


14 


284.00 


.000 


.246 


Hotellings 


.908 


9.084 


14 


280.00 


.000 


.312 


Wilks 


.519 


7.827 


14 


282.00 


.000 


.280 


Roys 


.470 












Test for Four Secondary Financial Variables 










Pillais 


.414 


1.635 


28 


396.00 


.024 


.104 


Hotellings 


.485 


1.638 


28 


378.00 


.023 


.108 


Wilks 


.639 


1.641 


28 


347.56 


.023 


.106 


Roys 


.186 













Implications and Recommendations for Research and Practice in HRD 

The increased emphasis on and examination of individual, team, and organizational learning practices have 
stimulated tremendous interest in the concept of the learning organization. Although the concept is fairly well 
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established, it is still evolving and a certain amount of confusion and ambiguity surrounds it (Leitch, Harrison, 
Burgoyne & Blantem, 1996). Watkins and Marsickfe (1993, 1996a, 1996b) DLOQ instrument studies have shown 
moderate evidence of construct validity for the scales measuring the dimensions of the learning organization. 
However, the researchers have acknowledged that additional studies are needed to cross validate the instrument 
with different populations of organizations and with larger samples in order to more firmly establish its utility and 
validity. This study further validates the instrument in a new context. 

From an empirical perspective, the relationship between the presence and development of practices 
associated with the learning organization concept and organizational performance has not been adequately 
established. 

To assess this association, the current research integrates objective measures of firm performance obtained from the 
COMPUSTAT and the Stern Stewart Performance 1000 financial databases. This aspect of the study represents a 
unique methodology that, to date, has not been employed studies associated with the learning organization concept. 
Accordingly, our research findings suggest that the learning organization concept may positively associated with 
firm performance. Future studies should further investigate these exploratory results using a wide variety of 
financial indicators in different contexts. 

There are several limitations associated with this research. The sample, although randomly drawn, 
included only firms for which secondary data was available. Different results may have been obtained if smaller, 
privately-owned firms were also included in the sample. This study includes only a limited number of secondary 
financial performance measures to assess the relationship between the dimensions of the learning organization 
construct and financial performance. The inclusion of other financial measures may have yielded different results. 
Perceptions of upper-level managers and front-line employees were not solicited for this study. It is possible that 
different results may be obtained by soliciting a larger, more holistic sampling strategy within each firm to assess 
the organization relative to the DLOQ. Each of these limitations, however, represent opportunities for future 
research in this area. 

In conclusion, it has been acknowledged that creating learning organizations requires new roles for 
managers, human resource developers, and employees in building the capacity for learning at the individual, team, 
and organizational levels. Yet, there has been little empirical research to support the claim that performance 
improvement is related to the adoption of practices associated with the learning organization concept. Our 
exploratory research lends credence to the efficacy of the learning organization concept by suggesting that there 
may be a positive association between the learning organization concept and firm performance. 
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Translation, Validation, and Adaptation of an Instrument to Assess Learning Activities in 
the Organization: The Spanish Version of the Modified “Dimensions of the Learning 
Organization Questionnaire” 
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This paper shares methodology and reports findings from a translation , validation and adaptation 
study of the Dimensions of the Learning Organization Questionnaire (DLOQ). Results indicate 
that the Spanish version of the DLOQ seems to be a valid instrument to assess learning activities 
in organizations with Spanish speaking populations, especially those of Latin America. 



It can hardly be denied that technology, along with science, has historically been among the central engines of 
modernity. It is equally undeniable that the modem world is increasingly becoming a technological world. 
Technology is, by all indications, going to become the distinguishing feature of global culture in the coming 
decades. The business literature generally refers to at least three types of technology: product, process and 
management. Product technology is the knowledge used to produce any product - the information that specifies the 
product's characteristics and uses. Process technology is the knowledge used in production to organize the inputs 
and operate the machinery. It relates to processes by which a given product or service is produced. Management 
technology is the knowledge used in operating a business-the managerial skills that enable a firm to compete by 
using its resources effectively. Management innovation, like product and process innovation, depends on new 
technology. New technology for management, as for engineering, comes in the form of new knowledge, tools and 
methods (Stata, 1989). 

A country cannot develop a highly productive network of production in manufacturing, agriculture, 
transportation and communication, or other sectors except by a progressive evolution of an industrial structure in 
which technological advances play a fundamental role. If a country^ advance in technology continues long enough, 
it will probably reach a stage at which here and there, in one field or another, it develops technologies beyond those 
known in the most advanced countries. Until it reaches this stage, its technological advance will consist in large part 
of the adaptation of technologies already in use in the technologically advanced countries. This process is described 
as the ‘transfer” or ‘transmission” of technology from the developed countries (DCk) to the less developed 
countries (LDCk) (Hagen, 1980). 

The transfer of technology from the technologically developed world is a vital approach to bringing 
sophisticated technology to an LDC. Technology transfer is an important medium for generating a more efficient 
modem technology in a developing country. In this case, transfer of technology involves taking some equipment, 
techniques, and practices developed in some technologically advanced country to some LDC. The assumption or 
anticipation is that the local people in the developing country, will be able to acquire the techniques transferred to 
them. Acquiring techniques means being able to learn, understand, analyze, and explain the whys and hows of those 
techniques. It is also anticipated that the local technologist, who is the beneficiary of the transferred technology, 
will be able not only to adapt the received technology to suit the needs and circumstances of the developing country, 
but also to build on it and to use it as an inspiration to create new technologies appropriate to the development 
requirements and objectives of the developing country. 

There has been a widespread belief among developing countries that unless technology is properly 
transferred from the more industrially advanced countries, there could never be an adequate industrial and economic 
development in general. Latin American countries have long recognized a need to use and adapt foreign technology 
to local conditions, and to generate indigenous products, processes and practices (Marton, 1986). 

Industrial development has been the objective of most development programs in Latin America since the 
turn of the century. Although such development was often implicit rather than explicit in national efforts, the model 
of development has been based on industrial growth complemented by manufacturing, agricultural investment, and 
extraction of basic minerals. Many promising developments have been truncated by profound crises that have 
affected Latin American economies through the years. During the 1980s, for example, the entire region was 
plunged into deep recession and stagnation. Unprecedented inflation, unemployment, devaluation, and a sky-high 
foreign debt afflicted the region (World Press review, 1988). Compounding these economic problems was the 
dramatic political turmoil that ripped Latin American economies apart. Although some problems still remain, Latin 
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American countries are currently experiencing a revival brought about by new leadership and powerful external 
factors. A new generation of leaders are leading the states into progressive economic programs, are advising their 
governments, initiating proposals for deregulation, and encouraging greater regional integration ( Clerc, 1990; Baker 
& Weiner, 1992). External factors have also been a major trigger of the ongoing reforms. The trend toward 
globalization of markets has forced Latin American countries to replace their old economic thinking. The 
strengthening position of regional blocks and the emergence of newly industrialized countries have resulted in a 
more diverse and fierce competition all over the globe, and most leaders in the region are starting to realize the 
dangers of being left out (Graham, 1990). In order to carve a position in this global economy, Latin America with a 
market of about 450 million people, an estimated gross domestic product (GDP) of US $900 billion (1990 figures), 
and abundant human and natural resources, is being forced to implement long overdue changes in their economic 
and trade policies as well as to take initiative to foster the role of science and technology in economic and social 
development. Specific policies on the importation and the local development of technology have been adopted in 
many Latin American countries, particularly in Argentina, Brazil, Mexico and the Andean Group (Peru, Venezuela, 
Ecuador, Colombia and Bolivia). 

Recent studies suggest that the key to success for an organization is embodied in its ability to implement 
and appropriate new technology (Willmann, 1991). However, adopting new technology does not ensure its 
successful integration and its optimum use. A company must be able to adjust to the demands and opportunities the 
new technology creates in order to realize its full advantage. 

There is no simple formula for making the changes required to fully exploit a new technology. Indeed, 
every organization is unique and must identify and implement the changes required to enhance its own effectiveness. 
Argyris and Schon (1978) describe this process of identifying and implementing required changes as organizational 
learning. They suggest that an organization is, at its root, a cognitive enterprise and it learns and develops 
knowledge. The concept of a learning organization has evolved from the ideas of organizational learning, but it 
differs in that it includes not only the learning of the organization, but learning within the organization (Ulrich, Von 
Glinow & Jick, 1993). Learning organizations create intentional processes that accelerate the creation and 
utilization of knowledge across the system (Marsick & Watkins, 1997). Watkins and Marsick (1993; 1996)delineate 
the learning organization as one that captures, shares, and uses knowledge to change the way the organization 
responds to challenges. They describe seven dimensions or action imperatives necessary for organizations to 
progress toward becoming a learning organization: create continuous learning opportunities ( Continuous Learning ), 
promote dialogue and inquiry ( Dialogue and Inquiry ), promote collaboration and team learning ( Team Learning ), 
empower people toward a collective vision Empowerment ), establish systems to capture and share learning 
( Embedded System ), connect the organization to its environment ( System Connection ), and provide strategic 
leadership for learning ( Provide Leadership). 

These action imperatives form the basis for the Dimensions of the Learning Organization Questionnaire 
(DLOQ), a survey developed by Watkins & Marsick (1997) to assess learning activities in the organization. 
Watkins and Marsick added also additional items to the survey in order to measure Knowledge Performance which 
is defmed as: Enhancement of products and services because of learning and knowledge capacity; and Financial 
Performance which is defined as: State of financial health and resources available for growth. Several studies have 
already established reliability and content and predictive validity of the DLOQ with and without knowledge and 
financial performance scales (Watkins, Selden, & Marsick, 1997; Watkins, Yang, & Marsick, 1997; Yang, Watkins 
& Marsick, 1998; McHargue, 1999; Watkins & Marsick, 1999). 

In the context of this study, a learning organization is defined as the ultimate goal of a process that 
promotes continuous learning at the individual, group, and organizational levels in order to modify behavior and 
positively impact the organization^ ability to deal with the change that a technology transfer process brings about. 
The major premise is that those organizations which develop as learning organizations are better at adapting to 
change (Kline & Saunders, 1993; Marquardt & Reynolds, 1994; Nonaka & Takeuchi, 1995; Senge, 1990; Swieringa 
& Wierdsma, 1992; Watkins & Marsick, 1993, 1996; Marsick & Watkins, 1999). 

Much has been written to convince companies of the benefits of developing themselves as learning 
organizations, but little has been offered in terms of assessment tools and empirical data, particularly in a Latin 
American context. The DLOQ is a useful instrument that has consistently produced acceptable results in the United 
States, and may be used extensively in the areas of human resource and organizational development in Latin 
America. Hence, this study sought to translate, validate, and adapt the 7 dimensions of the learning organization and 
the knowledge performance scales so that could be used with Spanish-speaking populations. A method based on the 
four basic translation methods described by Brislin (1976, 1980) combined with additional activities was used, 
which consisted of 1) forward-translation, 2) assessment of forward translation for clarity, common language and 
cultural adequacy, 3) back-translation, and 4) assessment of back-translation for conceptual equivalence. 

According to Brislin, Lonner and Thorndike (1973), ‘Unless researchers present empirical evidence to 
support their claim that different language versions of the same instrument are equivalent, translation problems will 
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always be plausible rival hypotheses for any obtained results” (p.32). The equivalence of measurements is therefore 
one of the central problems of cross-cultural research (Hui & Triandis, 1983). If equivalence of translation can not 
be demonstrated between translations, it remains uncertain whether any differences noted between subjects could be 
attributed to differences in item translation, whether the translated scales are comparable to the original scale, and 
whether differing translations are equivalent in constructs addressed. 



Methods 

Sample 

In the present study, a purposive, non-random sample was used. The target population was medium to 
large size private manufacturing corporations located in Bogota, Colombia. The decision to choose this country was 
based on the following criteria: 

a) It is a Spanish speaking country. 

b) Lists of population elements from which a sample may be drawn were available 

c) Personal contacts were available. 

d) This country is a member of the most important trading block of the region: the Andean Pact. 

Furthermore, Colombia is one of Latin America's top performing economies. With a population of 36 
million people. Despite the fact that the mere mention of its name conjures up images of guerrilla movements 
wreaking chaos, Colombia has had positive economic growth every year since 1948, has a history of stable 
exchange rates, and has never experienced a period of hyper-inflation, has never defaulted on its sovereign debt 
obligations, and never asked to reschedule its payments. In addition, the Colombian military has a history of 
non-intervention in internal politics and Colombia has always respected foreign ownership. These positive points 
indicate that the Colombian market is one in which foreign and indigenous organizations have had and still have a 
real chance to develop and succeed. 

To be included in the sample, a company would have to meet several criteria: it must be 1) private, 2) 
engaged in manufacturing operations, 3) medium to large size (number of employees was the only metric used to 
determine its size.), and 4) be or have been a target of any form of technology transfer in the last 10 years. 

Once the target population to be sampled was determined, the availability of a list of population elements 
from which a sample may be drawn was assessed. In an international context, this frequently presents difficulties 
due to the paucity of information available on industries, business, or consumer groups in other countries. Even 
where sampling frames, such as electoral or municipal lists, directories, telephone books, or mailing lists commonly 
used in the United States are available, they frequently do not provide adequate coverage, particularly in less- 
developed countries, and hence, give rise to frame error. Worldwide or regional lists of manufacturers can be 
obtained from sources such as Bottin International. This source registers names and addresses of more than 300,000 
firms in 100 countries, under 1,000 product classifications, by trade and by country. 

In this particular study, two lists were used: a) The Latin American Companies Handbook, b) Latin 
America 25,000/Dun & Bradstreetfc key business directory. Thirty companies enlisted in both lists were selected 
and then information about the company itself was drawn. Initial contact with the company was by e-mail to the 
executive director, explaining the study and requesting authorization to survey employees at different levels within 
his/her organization. The names and a brief background of trained personal contacts in Bogota who were in charge 
of follow-ups on behalf of the researcher were given. Following this first contact, a hard copy of the electronic letter 
with a sample of the questions included in the SDLOQ was faxed to the same executive. The follow -up was done 
via telephone by a trained personal contact residing in Bogota, who coordinated and consulted with the researcher 
over the companies concerns via e-mail. Eight companies agreed to be part of the study. Data collection was done 
on site. The SDLOQ was distributed and administered directly by the researcher and a total of 906 valid responses 
were properly collected. 

Forward Translation 

The initial translation of the DLOQ from English to Spanish was performed by male and female bilingual 
translators whose mother tongue was the target language. The female translator was known to the researcher for 
her expertise in translation of documents and technical information. This individual is a professional translator, 
fluent in both languages and a lifelong member of the Latin American community in which the study was going to 
be conducted. The male translator was the researcher who is Peruvian-American, fluent in both languages and 
familiar with several Latin American cultures. Prior to beginning the translation procedure, explicit information 
regarding the use and intent of the DLOQ instrument was given to the female translator. Initial instructions included 
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a request to avoid focusing or centering on the use of Spanish terminology when completing the initial translation, to 
use common Spanish equivalents for all words and phrases, and to translate the original text as closely as possible. 

Assessment of Forward Translation 

The Spanish translations provided by the translators were then presented to a panel of four additional 
Hispanic individuals, with advanced degrees in different disciplines, knowledgeable of many Latin cultures, and 
originally from Argentina, Colombia, Peru and Venezuela. These individuals were asked to independently review 
each item of both translations and to choose the one which they judged to be the best in terms of clarity, i.e. the item 
must express a single idea, should not contain ambiguous terms, should be easy to read and understand; common 
language, i.e. the item is expressed with language used by the general population; and cultural adequacy, i.e. the 
item is appropriate and relevant for the culture for which it is being adapted. Following a discussion and agreement 
among the participants, a consensus translated version was developed consisting of item versions picked out from 
one or the other translation or a newly generated item. 

Backward Translation 

The consensus Spanish translation was then presented to an additional Hispanic bilingual proficient in both 
languages. This individual was requested to back translate the Spanish translation to English and to identify any 
corrections in grammar, word usage, or diacritical markings deemed necessary in the Spanish translation provided. 
The translator was an Argentinean-American female, whose native language was Spanish and who held a Bachelor^ 
degree in English and a Master^ degree in Spanish Linguistics. 

Assessment of Backward Translation 

Following this step, the back translation was reviewed by a panel which consisted of one of the developers 
of the DLOQ, a methodologist, and the researcher. The review process focused on the conceptual equivalence with 
the original version of the DLOQ. Item by item was reviewed and decisions on equivalence were based on the 
following options: a) both the wordings and the meanings have nothing to do with each other; b) the wordings are 
equivalent in some respects but they do not capture the same meaning; c) the wordings are not equivalent but they 
capture the same meaning; and d) the wordings are equivalent and capture the same meaning. Ninety five percent of 
items were found to be conceptually equivalent, 2 modifications in wording of the Spanish translation were 
proposed until an acceptable compromise was reached and a final version was approved. 



Results 

The translation technique used in this study aimed at putting the DLOQ in Spanish while preserving the same ideas 
across the linguistic and cultural boundaries. Item equivalence and conceptual equivalence was established by 
comparison of meaning between original and back-translated forms and comparison of meanings between the 
original and translated form, by bilinguals other than the translator ( Brislin, et. al., 1973). It should be reported, 
though, that some conceptual discrepancies were identify in the back translated version of the instrument due to the 
meaning of the words ‘trust” and ‘Empower.” In the forward translation the word ‘trust” was translated as 
“confianza” and ‘Empower” as “ autorizar.” Obviously, when these words were back translated they resulted in 
‘fconfidence,” which is different from ‘trust” and ‘Authorize” which is different from ‘Empower.” Cultural 
considerations were necessary to consider in order to arrived at an agreement in terms of conceptual equivalence. 
This process was expedited due to the fact that the researcher is a native Spanish speaker, culturally indigenous 
(emic), and familiar with the theoretical underpinnings of the instrument in question. 

Statistical Analysis 

Cronbachfc Alpha coefficients for each dimension of the scale ranged from .791 to .835 on the SDLOQ. 
Overall, the reliability estimate for the entire scale is .960 for the 49 item version of the translated instrument. Table 
1 below reports Alpha coefficients for each scale of this study (N = 906) and for previous USA studies. In all but 
except one case, the reliability estimate was comparable with the American studies. It appears that the Team 
Learning construct is not as tight a construct as it is in America or maybe, we could argue, that the reliability of this 
construct is curvilinear as the sample size increases. These Alpha coefficients, which are evidence of construct 
validity, suggest that the SDLOQ is a reliable instrument. 
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Table 1. Scale Reliability Scores 





American Studies 


Colombian Study 


Scale Names 


Alpha (N = 68) 


Alpha (N =191) 


Alpha (N = 469) 


Alpha (N = 906) 


Continuous Learning 


.74 


.82 


.79 


.80 


Inquiry & Dialogue 


.78 


.87 


.85 


.81 


Team Learning 


.88 


.86 


.84 


.79 


Embedded Systems 


.86 


.85 


.79 


.81 


Empowerment 


.83 


.85 


.82 


.81 


Systems Connections 


.76 


.85 


.75 


.80 


Provide Leadership 


.83 


.83 (N = 74) 


.86 


.84 


Knowledge Performance 


.82 


.78 (N = 61) 


.74 


.82 



Predictive validity refers to the extent to which the instrument scores predict criterion measurement 
(Crocker & Algina, 1986). It is reasoned that the measurement^ predictive validity will be shown if scores on the 
dimensions of the learning organization can successfully predict knowledge performance. In order to determine the 
predictive validity of the SDLOQ in terms of knowledge performance, multiple regression analysis was conducted 
where the measure of knowledge performance was designated as a response variable and scores on seven 
dimensions of learning organization were treated as predictor variables. The regression analysis results indicated 
that the multiple correlation between learning activities measured on seven dimensions of the SDLOQ and the 
measure of knowledge performance was statistically significant (p<001). The estimated multiple correlation 
coefficient was .616 and the R-square was .380. This means that 38 percent of the variation in knowledge 
performance can be explained by the organizational learning activities measured on the SDLOQ. That is to say, the 
SDLOQ showed strong predictive validity in predicting knowledge performance in the organization. Table 2 below 
reports the R-square for this and previous studies. 



Table 2 . R- Square 



Study 


Watkins et al., 1997 


Yanget al., 1998 


McHargue, 1999 


Present 


Sample Size 


64 


469 


246 


906 


R' 


.324 


.63 


.260 


.38 


_E5 


.05 


.01 


.01 


.001 



Conclusions 

The SDLOQ has revealed similar characteristics, in terms of statistical properties, to the original; in fact, in some 
instances, results have been even better than those registered with the original one (e.g. Cronbachk Alpha). 
Therefore, and as with the original scale, the SDLOQ seems to be a valid instrument to assess learning activities in 
organizations with Spanish speaking populations. 

The present study has provided Latin American researchers and HROD practitioners with a Spanish version 
of the Dimensions of the Learning Organization Questionnaire (SDLOQ), an instrument that will assist them in 
generating data to assess the actual state of Latin American organizations with respect to the goal of becoming 
learning organizations. 
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